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TRAINING  FOR  DYNAMIC  AEROSPACE  CONTROL: 

AN  EXPERIMENT  IN  INTERNATIONAL  DISTRIBUTED  MISSION  TRAINING 


INTRODUCTION 

Current  training  resources  and  methods  limit  our  ability  to  train  and  practice  combined  air  operations 
involving  coalition  forces.  Distributed  Mission  Training  (DMT)  offers  new  opportunities  to  provide  our 
air  forces  with  the  knowledge,  skills,  and  experiences  needed  to  successfully  train  and  practice  such 
missions.  In  response  to  these  opportunities,  the  Air  Force  Research  Laboratory,  Human  Effectiveness 
Directorate,  Warfighter  Training  Research  Division  (AFRL/HEA)  and  Thales  Training  and  Simulation 
(TTS)  established  a  Cooperative  Research  and  Development  Program  to  investigate  the  potential  of  DMT 
and  other  advanced  distributed  simulation  technologies  as  an  additional  media  for  training  coalition  air 
operations.  This  program,  known  as  Project  Allied  CRAFT,  seeks  to  establish  a  technical  base  that  will 
assist  the  United  Kingdom  and  the  United  States,  as  well  as  their  allies,  in  developing  a  multi-national 
simulation  network  to  enhance  mission  training. 

Project  Allied  CRAFT  is  designed  to  use  the  distributed  simulation  capabilities  and  supporting 
infrastructures  in  the  United  Kingdom,  United  States,  and  the  North  Atlantic  Treaty  Organization 
(NATO)  and  to  grow  as  these  capabilities  evolve.  It  is  planned  as  a  three-phase  effort.  Budget  realities, 
resource  availability,  and  classification  issues  will  ultimately  determine  if  all  three  phases  are  successfully 
completed.  The  phases  are: 

•  Phase  I,  Training  for  Dynamic  Aerospace  Control,  was  intended  to  provide  confirmation  of  the 
international  DMT  concept  in  the  form  of  a  training  demonstration  to  show  current  technology 
has  the  potential  to  solve  many  of  the  operational  training  problems.  Phase  I  was  completed  1 1- 
13  September  2000  as  part  of  Air  Force  Association’s  2000  Aerospace  Technology  Exposition 
and  is  described  in  this  report. 

•  Phase  II  will  consist  of  a  research  and  development  (R&D)  program  to  increase  capability  and 
conduct  training  research  experiments  to  define  the  technical  and  training  requirements  needed  to 
support  and  justify  DMT  procurement  action. 

•  Phase  III  will  establish  an  International  DMT  Program,  possibly  on  a  Joint  Integrated  Product 
Team  0PT)  basis.  It  would  no  doubt  include  North  Atlantic  Treaty  Organization  NATO) 
ground-based  training  assets  as  they  are  developed  and  fielded. 

The  primary  purpose  of  this  paper  is  to  document  the  results  of  the  Phase  I  training  technology  concept 
demonstration.  This  demonstration,  Training  for  Dynamic  Aerospace  Control,  linked  multiple  sites  using 
the  Internet  and  commercia  1  data  lines.  We  begin  with  a  brief  discussion  of  the  background  for  the 
project.  This  discussion  is  followed  by  identification  of  our  goals  and  objectives  and  a  brief  synopsis  of 
our  earlier  unsuccessful  attempt  to  link  with  TTS  in  1999.  Following  the  background  material,  we 
describe  the  successful  demonstration  conducted  in  September  2000  and  some  of  the  technical 
performance  data  collected  during  that  demonstration.  Finally,  we  conclude  with  a  discussion  of  the 
lessons  learned  from  the  Phase  I  demonstration  and  the  potential  for  distributed  simulation  to 
revolutionize  the  way  we  prepare  for  coalition  operations. 

BACKGROUND 

Recent  military  conflicts  have  been  predominately  “asymmetric.”  This  asymmetry  stems  from  the  use  of 
high  technology  systems  against  relatively  simple  and  widely  dispersed  enemy  forces  that  often  lack  well- 
established,  military- industrial  infrastructures.  Because  of  the  lack  of  well-defined  enemy  lines  and 
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military-industrial  infrastructure,  the  control  of  aerospace  operations  is  becoming  increasingly  dynamic. 
Small,  highly  mobile  enemy  targets  must  be  quickly  pinpointed  and  aerospace  assets  rapidly  deployed  to 
successfully  attack  such  targets  with  minimal  collateral  damage. 

Training  aerospace  forces  to  successfully  Find,  Fix,  Track,  Target,  Engage,  and  Assess  damage  in  this 
time -compressed,  highly  dynamic  mission  environment  presents  a  major  training  challenge.  Because 
these  missions  involve  a  complex  system  of  systems,  there  are  numerous  individuals  and  teams  that  must 
interact  with  one  another  to  successfully  plan  and  execute  each  mission.  Unfortunately,  these  teams  have 
limited  opportunities  to  train  together  in  realistic,  collective  training  environments.  These  limited  training 
opportunities  mean  that,  while  these  teams  may  have  developed  high  levels  of  expertise  for  the  individual 
and  team  tasks  within  their  particular  specialties,  they  may  not  have  similar  levels  of  expertise  involving 
the  inter-team  tasks  needed  to  effectively  operate  as  a  part  of  a  larger  dynamic  aerospace  force.  This 
training  problem  is  further  exacerbated  by  the  increasing  reliance  on  multi-national,  coalition  forces  who 
have  even  fewer  opportunities  to  conduct  multi-team  collective  training  prior  to  a  real  conflict. 

One  potential  solution  to  this  training  challenge  is  to  use  DMT  to  create  a  distributed  virtual  training 
environment  in  which  the  various  teams  and  nations  can  collectively  train  together  in  a  common  virtual 
environment  from  home  station.  Such  a  virtual  environment  could  supply  realistic  mission  training  for 
national  and  multi-national  teams.  However,  development  and  use  of  such  virtual  environments  is  still  in 
its  infancy.  As  a  result,  there  is  limited  data  available  to  operational  warfighters,  engineers,  and  training 
developers  that  describe  the  current  readiness  of  both  simulation  technologies  and  training  methods  to 
provide  such  training.  Consequently,  it  is  difficult  to  determine  the  capability  of  these  technologies  and 
methods  to  meet  warfighter  training  needs  and  to  identify  the  enabling  technologies  that  must  be  further 
developed  to  provide  a  robust  team  training  environment.  This  paper  describes  a  demonstration  that 
linked  simulators  located  in  the  United  States  and  the  United  Kingdom  to  conduct  a  virtual  training 
exercise  and  the  data  collected  during  that  demonstration. 

Goals  and  Objectives 

There  were  two  goals  for  the  Phase  1  demonstration.  The  first  goal  was  to  demonstrate  an  international 
DMT  capability  between  the  United  States  and  the  United  Kingdom  using  currently  available  products 
and  technology.  The  second  goal  was  to  determine  the  potential  utility  of  including  command,  control, 
intelligence,  surveillance,  and  reconnaissance  (C2ISR)  assets.  Effective  integration  of  C2ISR  assets  into 
a  real-time,  international  distributed  simulation  network  could  significantly  enhance  its  training  value  by 
providing  the  capability  to  train  the  entire  sensor-to-shooter  mission.  Such  missions  involve  a  number  of 
the  different  teams  that  need  to  work  together  to  successfully  plan  and  execute  coalition  air  operations. 

Based  on  these  goals,  the  following  objectives  were  established  for  Phase  I  of  Allied  CRAFT. 

•  Showcase  existing  technology  for  Distributed  Mission  Training 

•  Establish  a  wide-area,  international  DMT  connection  to  the  United  Kingdom 

•  Successfully  execute  a  number  of  complex  mission  scenarios  using  that  wide-area,  international 
network 

•  Determine  the  utility  of  using  real-time  distributed  networks  to  augment  coalition  training 

•  Document  the  latencies  involved  with  transoceanic  networks 

•  Identify  the  bandwidth  requirements  of  a  composite  force  mission  scenario  executed  in  real-time 

•  Demonstrate  the  use  of  the  Internet  as  a  possible  low-cost  method  for  providing  DMT  training 

•  Document  what  worked  and  what  didn’t  work  during  the  international  DMT  demonstration. 
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1999  Air  Force  Association  Technology  Exposition 

The  first  attempt  to  complete  Phase  I  of  Allied  CRAFT  was  in  conjunction  with  the  1999  Air  Force 
Association  (AFA)  Technology  Exposition  in  Washington  DC.  Our  original  plan  called  for  the 
establishment  of  a  wide-area-network  (WAN)  that  linked  a  number  of  sites  in  the  US  and  TTS  in  the  UK. 
These  sites  and  the  various  virtual  and  constructive  simulations  from  each  of  those  sites  are  shown  in 
Table  1. 


Table  1.  Simulation  Participants  Planned  for  the  1999  AFA  Technology  Exposition. 


Eocation 

Virtual  Simulators 

Constructive  Simulations 

Brooks  AFB,  TX 

E-3  Mission  Crew 

Hurlburt  Field,  FL 

AC-130U 

Tucson,  AZ 

1  x  A-10 

Crawley,  UK 

2  x  Tornado 

Washington,  DC 

2  x  F-16 

ZSU-23*  (Army  TSMO) 

Mesa,  AZ 

2  x  F-16 

Friendly  Aircraft 

1  x  A-10 

Enemy  Aircraft 

Enemy  Ground  Forces 

Kelly  AFB,  TX  Surface  to  Air  Threats 

*  ZSU-23  was  located  in  a  separate  booth  at  the  exposition  and  was  treated  as  a  separate  site 


Figure  1  shows  the  planned  WAN  design  for  the  1999  demonstration.  This  network  plan  assumed 
installation  of  a  dedicated  T-l  data  line  between  AFRL  Mesa  and  the  Washington  DC  show  site,  existing, 
dedicated  T-l  data  lines  between  Mesa  and  the  other  sites,  and  a  primary  rate  Integrated  Services  Digital 
Network  (ISDN)  between  Mesa  and  Crawley,  UK. 

Unfortunately,  we  were  unable  to  execute  the  1999  technology  demonstration  as  planned.  While 
preparing  to  deploy  to  Washington,  DC  the  week  before  the  AFA  Technology  Exposition  was  due  to 
begin,  the  telephone  companies  had  not  yet  installed  either  the  T- 1  or  ISDN  data  lines.  Based  upon  a 
series  of  verbal  discussions  with  the  telephone  companies,  we  speculated  that  the  lines  would  be  installed 
in  time  for  the  exposition  and  deployed  our  simulators  to  Washington.  Unfortunately,  such  was  not  the 
case.  Upon  arriving  in  Washington,  we  discovered  that  the  communications  infrastructure  was  being 
upgraded  throughout  the  Washington  area  and  that  there  would  be  no  available  T-l  lines  until  the  latter 
part  of  2000.  The  only  option  left  was  to  attempt  to  use  the  Internet.  Therefore,  we  leased  an  Internet 
connection  from  the  Marriott  Hotel  and  attempted  to  use  it  as  the  major  communications  pipeline  for  the 
1999  technology  demonstration. 

The  use  of  the  Internet  presented  two  immediate  and  major  problems: 

•  All  the  data  between  sites  had  to  be  encrypted 

•  The  leased  Internet  pipe  had  a  capacity  of  only  128  kilobytes  per  second. 
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Figure  1.  AFRL’s  1999  AFA  Technology  Exposition  DMT  Demonstration 


Based  on  these  limitations,  the  plan  to  connect  with  TTS  as  part  of  the  demonstration  was  cancelled.  The 
two-ship  demonstration  that  was  planned  to  be  virtual  RAF  Tornados  was  replaced  with  a  two-ship  of 
constructive  RAF  Tornados  generated  locally  on  the  network.  This  two-ship  was  given  the  same  airfield 
target  as  their  objective.  The  other  effects  generated  by  the  lack  of  a  T-l  line  and  the  narrow  bandwidth 
pipe  from  the  hotel  to  the  Internet  Service  Provider  (ISP)  for  the  Marriott  are  discussed  in  more  detail  in 
the  Results  section. 

Although  we  were  unable  to  establish  a  bi-lateral  connection  between  AFRL/HEA  and  TTS  for  the  1999 
demonstration,  we  did  leam  three  valuable  lessons  from  this  effort: 

•  First  we  discovered  that  we  could  successfully  execute  real-time  mission  simulations  involving  a 
number  of  geographically  dispersed  sites  in  the  US  using  the  Internet  as  one  of  the  major  trunks. 

•  Second,  we  showed  that  it  is  possible  to  use  both  a  point-to-point  encryption  device  and  the  Internet 
with  DIS  multicast  protocols. 

•  Third,  we  learned  that  the  commercial  communications  infrastructure  in  the  US  is  undergoing  such 
continuous  growth  that  may  not  be  possible  to  obtain  new  dedicated  data  lines  in  a  timely  manner  in 
every  case. 

2000  AFA  Technology  Exposition 
Overview 

The  1999  AFA  Technology  Exposition  provided  us  with  invaluable  information  regarding  the  difficulties 
of  obtaining  data  lines  in  a  large  metropolitan  area  and  on  the  use  of  the  Internet  as  a  means  of 
exchanging  real-time  simulation  data.  Based  on  knowledge  that  we  not  obtain  a  T-l  line  in  time  for  the 
AFA  2000  Technology  Exposition,  just  as  in  1999,  we  began  planning  for  the  2000  show  in  October  of 
1999,  one  month  after  the  we  returned  from  the  Washington  DC  show.  As  part  of  this  plan,  we  made  the 
early  decision  to  use  the  Internet  between  the  Marriott  Wardman  Park  Hotel  and  the  Air  Force  Research 
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Laboratory  (AFRL)  in  Mesa,  AZ,  for  the  2000  technology  demonstration.  Because  of  the  performance 
limitations  observed  in  1999  due  to  the  128  kilobit  per  second  wide  connection  from  the  hotel  to  their 
ISP,  we  negotiated  with  the  Marriott  to  lease  a  private,  dedicated  1- megabit/second  connection  from  the 
Marriott  to  its  ISP  for  use  during  the  September  2000  AFA  show.  In  January  2000,  we  finally  obtained  a 
dedicated  primary  rate  ISDN  line  (T-l  equivalent  of  1.5  megabit/sec  bandwidth)  between  AFRL  Mesa 
and  TTS  in  Crawley,  England.  Several  quick  engineering  tests  were  conducted  using  Distributed 
Interactive  Simulation  (DIS)  protocols  to  verify  interoperability,  determine  a  common  method  of  time- 
stamping,  and  determine  the  average  latency  between  Mesa  and  Crawley 

Once  we  had  established  our  primary  communication  lines,  we  began  refining  our  mission  scenarios  and 
finalizing  the  actual  simulations  that  would  be  part  of  our  2000  technical  experiment.  The  scenarios 
focused  on  distributed  mission  training  for  dynamic  aerospace  control  with  the  emphasis  on  time -critical 
targeting.  The  objective  was  to  demonstrate  the  utility  of  using  DMT  and  for  inter-team  training 
involving  C2ISR  assets,  decision  makers,  and  shooters.  The  goal  of  this  inter-team  training  was  to 
successfully  accomplish  the  six  stages  of  the  “kill  chain”  -  Find,  Fix,  Track,  Target,  Engage,  and  Assess. 

We  decided  that  there  would  be  three  primary  WAN  nodes  for  the  AFA  2000  Technology  Exposition. 
These  nodes  were  the  Marriott  Wardman  Park  Hotel,  Washington,  DC;  TTS,  Crawley,  England;  and 
AFRL/HEA,  Mesa,  AZ.  Table  2  shows  the  simulators  and  simulations  associated  with  each  of  these  sites.. 


Table  2.  Simulation  Participants  for  the  2000  AFA  Technology  Exposition. 


Location 

Virtual  Simulators 

Constructive  Simulations 

Marriott  Wardman  Park, 

Combined  Aerospace 

AirSF  intelligent  agent 

Washington,  DC 

Operations  Center  (CAOC) 

1  xF-16 

1  x  A-10 

Joint  Stars  Mission  Simulator 

(called  VSTARS) 

AWACS  E-3B  Weapons 
Director 

Space  Maneuver  Vehicle 

Predator  Uninhabited  Air 
Vehicle 

constructive  force  generator, 

AFRL/HEA,  Mesa,  AZ 

3  xF-16 

Friendly  and  enemy  air  and 

1  x  A-10 

ground  forces 

TTS,  Crawley,  England 

1  x  Tornado 

1  x  Tornado 

Scenario 

The  scenario  used  in  the  2000  AFA  Technology  Exposition  was  a  modified  version  of  the  1999 
demonstration  still  conducted  on  the  Nellis  Range  Complex.  Figure  2  illustrates  the  WAN  configuration 
for  the  2000  demonstration,  and  Figure  3  shows  the  initial  conditions  for  this  scenario  at  “time  zero.” 
From  a  mission  perspective,  the  scenario  included  offensive  counterair,  interdiction  (conventional  strike), 
high-value  asset  cap,  suppression  of  enemy  air  defense  (SEAD),  time-critical  targeting,  battle 
management  from  airborne  warning  and  control  system  (AW ACS)  and  Joint  Surveillance  Target  Attack 
Radar  System  (JSTARS),  and  reconnaissance  from  the  Space  Maneuvering  Vehicle  concept  demonstrator 
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and  the  UAV  Predator.  The  theme  of  the  demonstration  was  Training  for  Dynamic  Aerospace  Control, 
which  focused  on  (and  demonstrated)  the  three  major  categories  of  execution  in  an  Air  Operations  Center: 

•  Fixed  Target  -  Planned  Event  (Aircraft  on  an  Air  Tasking  Order  (ATO)  tasked  sortie) 

•  Planned  Dynamic  -  Known  in  advance,  but  unable  to  fix  location  in  advance  (e.g.,  Moving 
Tanks) 

•  Unplanned  Dynamic  -  Not  known  in  advance  (e.g.,  SCUD  TEL  or  Launch) 


AFA  2000  Simulation  Nodes 


Thales  Training 
&  Simulation 
Crawley  UK 


=  Primary  Rate  ISDN 
=  Internet  Connection 


Figure  2.  AFRL’s  2000  AFA  Technology  Exposition  DMT  Demonstration 


Because  it  is  difficult  to  hold  the  attention  of  an  audience  very  long  in  a  technology  demonstration,  the 
scenario  was  designed  to  be  as  short  as  possible  and  have  as  much  tactical  action  as  practical  without 
sacrificing  the  demonstration  of  training  utility  or  becoming  unrealistic  to  the  warfighters  executing  the 
mission.  Each  live  demonstration  began  with  the  players  at  a  point  just  prior  to  the  Forward  Edge  of 
Battle  Area  (FEBA),  except  for  the  UAV,  which  was  flying  behind  the  enemy  lines.  The  following 
describes  a  typical  scenario  evolved  from  the  initial  conditions  shown  in  Figure  3. 

The  mission  began  with  the  E16  four-ship  on  a  medium  altitude  ingress  followed  by  the  two  Royal  Air 
Force  (RAF)  Tornados  in  2-minute  trail.  These  six  strike  aircraft  were  executing  a  planned  ATO-tasked 
mission  to  strike  an  enemy  airfield  and  fuel  tanks  located  on  that  airfield.  One  set  of  constructive  or 
computer-controlled  F-15Cs  maintained  a  high  value  asset  barrier  cap  between  the  FEBA  and  the 
AW  ACS  aircraft.  A  second  set  of  constructive  F15Cs  proceeded  out  ahead  of  the  strike  package  on  a 
sweep  mission  to  engage  any  enemy  aircraft  considered  to  be  a  threat  to  the  friendly  assets.  Constructive 
F-16CJs  also  began  just  east  of  the  FEBA  in  a  SEAD  role.  After  all  active  emitters  were  suppressed,  the 
F-16CJs  returned  to  a  predetermined  cap  to  re-engage  any  new  threats.  The  two  virtual  A- 10s  (one  in 
Washington  DC,  one  in  Mesa)  proceeded  to  the  FEBA  to  await  tasking  on  any  new  moving  targets 
detected  by  airborne  sensors.  Finally,  several  groups  of  friendly  tanks  (constructive)  crossed  the  FEBA 
and  proceeded  west  to  engage  enemy  ground  forces. 
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AFA  2000  Scenario 


Figure  3.  AFRL’s  2000  AFA  Technology  Exposition  DMT  Demonstration  Scenario 

The  enemy  order  of  battle  west  of  the  FEBA  included  several  columns  of  tanks  up  and  down  the  FEBA, 
anti-aircraft  artillery  (AAA),  surface-to-air  missile  sites,  SCUD  Tels  being  transported  to  new  locations, 
MiG- 29s,  and  Su-27s.  As  the  mission  came  off  freeze,  the  Su-27s  came  out  of  orbit  in  the  south  and 
proceeded  east  where  the  F15C  sweep  aircraft  engaged  them.  Several  minutes  later,  once  the  enemy 
determined  that  strikers  were  inbound,  MiG- 29s  took  off  from  the  enemy  airfield  in  the  north.  These 
aircraft  were  engaged  by  two  of  the  F16Cs  inbound  on  a  strike  mission  to  that  airfield  being  controlled 
by  AW  ACS.  Several  minutes  into  the  mission,  the  VSTARS  simulator  detected  a  column  of  moving 
entities  heading  east  and  several  suspicious  movers  heading  south  along  the  FEBA.  VSTARS  tasked  the 
UAV  ground  station  to  redirect  the  UAV  to  the  area  of  the  suspicious  movers  to  confirm  that  they  were 
SCUD  missile  launchers.  AWACS  coordinated  with  the  VSTARS  aircraft,  and  the  A- 10s  were  told  by 
AWACS  to  proceed  towards  what  is  likely  a  column  of  moving  tanks.  The  A- 10s  identified  the  tanks  and 
proceeded  to  engage  them  with  Maverick  missiles.  The  UAV  confirmed  SCUD  launchers  and  passed  that 
information  back  to  VSTARS.  AWACS  directed  Viper  1  and  2  to  stand  by  for  a  divert  mission  and  were 
sent  to  VSTARS  frequency.  Viper  3/4  and  Lancer  31/32  (the  RAF  Tornados)  were  told  to  continue 
inbound  to  the  airfield  target.  The  VSTARS  authenticated  with  Viper  1  and  2  and  delivered  the  9-line 
divert  message.  Viper  1  and  2  then  proceeded  to  the  area  of  the  UAV,  identified  the  missile  launchers 
and  proceeded  to  destroy  the  SCUDs.  Meanwhile,  AWACS  detected  the  launch  of  the  MiG-29s  from  the 
target  airfield.  Vipers  3  and  4  were  paired  against  the  MiGs.  They  engaged  and  destroyed  the  enemy 
aircraft  and  proceeded  back  towards  the  target  airfield.  The  Tornados  continued  inbound  to  the  target 
airfield.  The  Tornados  struck  the  oil  tanks,  followed  by  Vipers  3  and  4,  who  returned  to  the  airfield  from 
the  air-to-air  engagement  and  delivered  their  conventional  ordnance  on  the  airfield.  The  airfield  was 
protected  by  heavy  AAA  fire  but  the  mission  was  successful.  At  this  point,  the  demonstration  was 
terminated  by  exercise  control. 

Marriott  Wardman  Park  Layout 

Figures  4  and  5  depict  the  floor  layout  of  the  two  booths  constructed  at  the  AFA  2000  show  site.  Booth 
3605  (the  main  hub  of  the  demonstration)  contained  all  of  the  C2ISR  simulations.  The  centerpiece  of 
Booth  3605  was  AFRL’s  modular,  deployable  video  wall.  It  consisted  of  six  67-inch  diagonal  screens 
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arranged  in  a  2-high  by  3- wide  configuration.  Each  cube  contained  a  Barco  808S,  color,  CRT-based 
projector. 


Figure  4.  AFRL’s  Booth  3605  at  AFA  2000  Technology  Exposition 

The  video  wall  served  as  a  mini-section  of  a  Combined  Air  Operations  Center  (CAOC).  The  Sound 
Storm  3D  four-channel  sound  system  with  speakers  located  at  all  four  comers  of  the  booth  complemented 
the  visual  presentation.  The  public  was  able  to  see  the  demonstration  visually  from  various  viewpoints 
provided  by  front-channel  video  from  the  F-16  and  A- 10  at  the  show  site  and  three  stealth  viewers  used  to 
move  about  the  battlespace.  The  audience  could  also  hear  all  of  the  C2  and  pilot  UHF/VHF  radio 
transmissions  throughout  the  demonstration  through  the  four-channel  sound  system.  AFRE  also  deployed 
a  distributed  briefing/debrie  fing  system  that  linked  the  AFA  show  site  to  the  testbed  in  Mesa.  The  flight 
lead  for  the  DMT  mission  briefed  from  Mesa  using  an  interactive  SmartBoard  and  Microsoft’s  Net 
Meeting  networked  over  a  separate  Internet  line.  Video  teleconference  (VTC)  equipment  was  integrated 
to  the  briefing  system  and  linked  over  three  additional  ISDN  lines  leased  from  the  Marriott.  The  VTC 
and  SmartBoard  systems  were  left  active  throughout  the  show  following  each  live  DMT  demonstration  so 
that  people  in  Washinton  DC  and  Mesa  could  observe  and  use  the  distributed  briefing  system  between 
shows. 

SGI  demonstrated  a  pre-production  system  called  the  Video  Capture  and  Exploitation  System  (VCES). 
This  system  was  hosted  on  the  new  Onyx  3000  on  loan  to  AFRE.  The  VCES  is  capable  of  accepting 
almost  any  input  signal  format,  capturing  one  frame  or  a  motion  sequence  of  frames  and  allowing  the  user 
to  totally  manipulate,  enhance,  or  annotate  the  captured  information  and  then  re-transmit  the  information 
as  desired  in  an  number  of  output  formats.  The  video  capture  system  was  displayed  on  a  two-channel 
visual  Reality  Center  system  Also  connected  to  the  Reality  Center  and  the  CAOC  video  wall  were  the 
VSTARS  virtual  simulator,  the  UAV  Predator  Milti-Task  Trainer  (MTT)  and  the  Space  Maneuvering 
Vehicle.  Mission-ready  aircrew  associated  with  these  platforms  operated  the  VSTARS  and  UAV.  The 
SMV  was  operated  by  one  of  the  developers  of  the  systems.  In  addition  to  the  above  training  devices  in 
Booth  3605,  an  E-3  AW  ACS  Weapons  Director  (WD)  simulator  was  also  on  the  network  operated  by  a 
WD  from  Tinker  AFB,  OK. 
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Booth  3605  was  located  about  75  feet  from  a  second  AFRL  display.  Booth  3629,  and  connected  to  it  by  a 
number  of  Category  5  Twisted  Pair  Ethernet  cables  and  Swire  RGB  coaxial  cables  strung  along  wire 
braces  suspended  from  the  ceiling  of  the  Marriott  convention  hall.  Figure  5  shows  the  layout  of  Booth 
3629.  This  booth  contained  the  F-16C  MTT  equipped  with  a  full  field-of-view  Mobile  Modular  Display 
for  Research  and  Training  (M2DART)  color  display  system  developed  by  AFRF  Mesa.  This  F16  was 
Viper  2,  wingman  of  the  flight  lead  located  in  Mesa.  An  A-10A  MTT  with  a  three-screen,  120-degree 
field-of-view  display  was  also  located  at  the  opposite  end  of  Booth  3629.  This  A- 10  was  Hawg  1,  the 
flight  lead  whose  wingman  was  located  in  the  DMT  Testbed  at  Mesa.  Both  cockpits  were  linked  via 
individual  network  interface  units  (NIUs)  to  the  Mission  Control  Center  (CAOC)  and  network  hub  in 
Booth  3605  shown  in  Figure  4.  SGI  Onyx2  Reality  Monsters,  using  the  high-fidelity,  photo-realistic 
development  database  from  AFRF’s  night  vision  training  system,  powered  both  cockpit  visual  systems. 

Network  Topology 

The  AFA  2000  demonstration  was  generated  from  three  primary  sites  with  multiple  simulations  running 
at  each  site.  Mission  Control  was  located  at  the  AFA  site.  As  described  in  detail  earlier,  the  three  nodes 
were  AFRF  in  Mesa,  AZ,  the  Marriott  Wardman  Park  Hotel  in  Washington  DC,  and  Thales  Simulation  & 
Training  in  Crawley,  England.  Figure  6,  the  Mesa  AZ  site;  Figure  7,  the  Washington  DC  site;  and  Figure 
8,  the  Crawley  England  site,  illustrate  the  hardware  configuration  at  each  of  the  three  locations. 
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PRI  ISDN 


Figure  6.  Mesa,  AZ  Site  Local  Area  Network  Topology 


In  preparation  for  the  AFA  1999  and  2000  shows,  all  original  hard  disk  storage  media  were  removed  from 
the  testbed  and  replaced  with  new  hard  drives  and  configured  with  unclassified  software.  While  the 
“clean  disk”  requirement  causes  initial  upfront  costs  associated  with  these  types  of  demonstrations  to  be 
higher  than  normal,  having  the  spare  unclassified  removable  storage  hardware  saves  significant  man-hour 
costs  on  any  subsequent  deployment,  demonstration,  or  operation  in  an  unclassified  environment. 
Stripping  and  rebuilding  operating  systems  is  very  labor  intensive  and  introduces  risk  to  the  software 
integrity  after  rebuilding.  The  ordnance  server  shown  in  Figure  6  was  there  to  provide  missile  flyout  for 
the  intelligent  agents  being  generated  by  AirSF.  As  can  be  seen,  two  gateways  to  the  outside  world  were 
used  in  the  Mesa  DMT  Testbed.  One  was  based  on  the  Motorola  NES  (Network  Encryption  System)  box 
used  to  connect  to  Washington  DC  via  the  Internet,  and  the  other  was  a  Mykotronix  KIV-7HS  used  to 
connect  via  the  ISDN  lines  to  Thales  Training  &  Simulation  site  in  England  (Figure  8).  Figure  7 
illustrates  the  local  area  network  topology  for  the  show  site  in  Washington  DC.  All  of  the  components  at 
the  show  site  were  equipped  with  a  network  interface  unit  (NIU)  that  connected  to  the  10/100  Base-T 
Switch.  AFRL  also  deploys  a  development  system  and  a  laptop  workstation  for  technology 
demonstrations  conducted  outside  of  Mesa,  AZ.  The  development  system  is  used  to  write  new  code  or 
modify  any  existing  source  code  and  then  recompile,  if  necessary.  The  laptop  workstation  is  used  to  boot 
all  simulators  over  Ethernet  from  one  location.  This  standardizes  the  software  load  and  streamlines  the 
boot  procedures  which  all  reduces  the  risk  of  failure  when  time  and  resources  are  critical.  The  use  of  a 
deployed  development  system  also  enables  the  engineers  to  have  access  to  debug  code.  This  improves 
their  capability  to  troubleshoot  whenever  complex  systems  are  deployed  and  reconstructed  in  less  than 
perfect  conditions. 
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Figure  7.  AFA  2000  Show  site,  Washington  DC,  Local  Area  Network  Topology 

Figure  8  illustrates  the  gateway  configuration  and  local  area  network  at  the  Thales  facility  in  England. 
Mesa  and  Crawley  were  connected  by  the  1.5- megabit  per  second  (ISDN)  lines.  An  ISDN  primary-rate 
line  with  thirty  64  kilobits  per  second  channels  was  established  to  provide  data  and  voice  communications 
between  the  two  locations.  Both  locations  used  Lucent  ISDN  multi-plexers  and  Cisco  2601  routers 
employing  forwarding  protocols  over  a  V.35  electrical  interface,  because  the  demonstration  was 
unclassified  but  connected  to  a  classified  facility  in  Mesa,  to  provide  a  firewall.  Each  end  of  the  ISDN 
connection  was  attached  to  a  Mykotronix  KIV-7HS  encryption  system  installed  to  operate  in  an 
unclassified  secure  mode  over  the  public  ISDN  network.  The  KIV-7HS  was  installed  between  the  Lucent 
multiplexer  and  the  Cisco  router.  Due  to  lessons  learned  from  the  engineering  test  conducted  in  February 
2000,  which  used  an  X.21  interface,  the  AFRL  and  TT&S  engineers  were  able  to  configure  a  non¬ 
standard  V.35  interface  within  less  then  an  hour  after  resolving  a  few  minor  problems. 


ISDN 


Mesa  UK 


Figure  8.  Crawley,  England  Site  Local  Area  Network  Topology 
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RESULTS 


As  was  mentioned  in  the  introduction,  AFRL  was  notified  on  the  Friday  before  the  1999  Exposition 
opened  that  there  would  not  be  a  T-l  line  available  and  that  the  west  coast  phone  company  involved 
would  be  unable  to  install  the  ISDN  needed  to  connect  the  United  Kingdom.  Friday,  Saturday,  and 
Sunday  before  the  show  in  1999  were  filled  with  engineering  challenges  and  high  levels  of  stress,  but 
creativity  prevailed.  AFRL  engineers  were  able  to  design  and  develop  a  software  solution  hosted  on  an 
SGI  02  that  enabled  DIS  protocols  which  are  a  multicast  to  be  processed  prior  to  encryption  into  a  unicast 
format  that  satisfied  both  the  encryption  operational  requirements  and  the  transmission  protocol 
requirements  of  the  Internet.  The  same  software  solution  then  reprocessed  the  packets  upon  receipt  after 
de-encryption  and  put  the  data  back  into  a  useable  DIS  PDU  multicast  format  for  each  local  area  network. 

As  stated  earlier,  the  bandwidth  available  between  the  hotel  and  the  Internet  connection  for  the  1999  AFA 
show  was  128  kilobits  per  second.  As  a  group,  we  assumed  that  the  narrow  bandwidth  Internet  connection 
would  not  allow  a  very  high  leve  1  of  activity  once  the  scenario  was  executed.  However,  we  had  no  choice 
but  to  use  the  narrow  connection  since  the  show  equipment  was  already  deployed  to  the  AFA  site.  The 
software  solution  was  finished  and  tested  late  Saturday  night  and  into  early  Sunday  by  setting  up  two 
NES  boxes  at  the  hotel  and  verifying  the  solution  appeared  stable.  Once  tested,  an  NES  box  and  the  new 
software  were  flown  back  to  AFRL  in  Mesa  for  installation  and  at  4:30  p.m.  on  Sunday  afternoon  (before 
the  Monday  opening)  the  DMT  Testbed  was  heard  on  the  UHF  radio  from  one  of  the  F-16s  in  Mesa  over 
the  Internet! 

The  use  of  the  narrow  pipe  to  the  ISP  in  the  1999  show  did  have  a  significant  impact  on  the  DMT  show. 
Routine  network  analyzer  equipment  confirmed  that  there  were  oftentimes  50%  to  100%  data  collisions 
occurring  depending  not  only  on  the  show  traffic  but  on  the  hotel  traffic  as  well.  While  everyone  was 
pleased  and  surprised  to  learn  that,  despite  the  narrow  data  path,  AFRL  was  able  to  conduct  the  live 
demonstrations  as  planned,  without  significant  degradation  in  the  visual  systems  or  weapons  employment, 
there  was  significant  impact  to  voice  communications.  Entities  from  different  sites  could  fly  formation 
with  each  other;  however,  if  they  attempted  to  fly  close  formation  then  the  result  of  the  numerous  data 
packet  collisions  was  apparent.  There  was  an  unacceptable  level  of  jitter  in  the  out- the -window  scene. 
Any  time  spent  outside  close  formation  was  acceptable.  The  major  impact  was  to  the  DIS  voice 
communications.  Whereas  a  missed  packet  of  positional  data  results  in  a  visual  frame  jump,  which  may 
or  may  not  be  noticed,  a  missing  (or  several  missing)  voice  packet  resulted  in  the  messages  being  chopped 
up  and  most  of  the  time  unreadable  or  unusable.  Since  this  problem  could  not  be  overcome  due  to  the 
narrow  bandwidth,  the  voice  channels  were  disconnected  from  the  public  address  side  of  the  booth  and 
AFRL  personnel  reverted  to  narrating  the  demonstrations  as  they  were  flown  each  time.  In  general,  most 
people  who  understood  the  challenge  AFRL  faced  in  1999  and  how  the  engineering  side  had  to  be  put 
together  appreciated  the  fact  that  we  had  demonstrated  that  the  Internet  could  be  used  at  all  for  such  a 
training  event. 

For  the  AFA  2000  Exposition,  AFRL  took  the  lessons  learned  from  the  1999  show  and  negotiated  with 
the  Marriott  Communications  Center  to  obtain  a  dedicated  1-megabit  per  second  trunk  to  the  hotel  ISP. 
Given  the  level  of  complexity  of  the  scenario,  this  solved  almost  100%  of  AFRL’s  problems.  Other  than 
latencies  no  doubt  introduced  by  the  use  of  extended  routing  algorithms  in  the  Internet  cloud,  there  was 
no  perceptible  impact  to  the  execution  of  the  DMT  demonstrations  including  voice  traffic.  The  AFA  2000 
DMT  demonstration  was  a  big  success  story  and  highlighted  the  fact  that,  aside  from  some  security 
issues,  the  Internet  may  offer  a  lower  cost  alternative  to  distributed  training  using  ground-based  systems. 

In  general,  the  data  packet  flow  rate  between  sites  varied,  as  expected,  based  on  the  complexity  of  the 
local  area  network.  The  total  number  of  entities  on  the  network  during  each  demonstration  varied  between 
95  and  120.  The  number  of  data  packets  per  second  (pps)  averaged  400  pps  with  peaks  around  600  pps. 
The  sustained  (over  a  5-min  period)  in-coming  bit  rate  to  Washington  DC  and  Crawley  UK  was  recorded 
as  800  Kbps.  This  would  occur  near  the  end  of  the  scenario  each  time  when  most  entities  were  engaging 
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and  firing  at  each  other  or  against  ground  targets.  There  were  occasional  peak  values  of  950  Kbps  when 
numerous  simultaneous  voice  transmissions  were  made.  The  ISDN  connection  between  the  US  and  the 
UK  systematic  round-trip  ping-time  was  variable  and  seemed  to  be  dependent  on  the  time  of  day  the 
phone  calls  were  placed.  The  average  latency  between  Mesa  and  Crawley  England  ranged  from  148  ms 
during  the  early  morning  calls  to  198  ms  during  the  calls  later  in  the  day.  It  should  be  mentioned  that 
there  was  a  6-hr  time  difference  between  Crawley  and  the  AFA  show  site.  The  average  latency  between 
Mesa  and  Washington  DC  over  the  Internet  was  80  ms. 

As  discussed  earlier  the  telecommunications  link  between  Mesa  and  Crawley  England  was  a  primary  rate 
ISDN  line.  Of  the  total  available  lines,  we  used  20  channels  for  each  session.  This  equates  to  1.28  Mbps. 
The  bit  stream  was  encrypted  using  the  KIV-7HS  interposed  between  the  ISDN  "modem"  and  router.  No 
noticeable  signal  delay  was  seen  through  the  encryptors.  Voice  traffic  between  Mesa  and  Crawley  was 
crystal  clear.  The  earlier  test  in  February  2000  highlighted  that  using  fewer  channels  than  20  resulted  in 
noticeable  voice  break-up.  The  sustained  (over  a  5-min  period)  out-going  bit  rates  from  Crawley 
averaged  approximately  20  Kbps  (1  virtual  Tornado  +  1  CGF  Tornado)  with  occasional  peaks  of  about 
100  Kbps  (including  voice).  The  ISDN  link-up  was  very  reliable,  easy  to  synchronize,  and  deemed  to  be 
extremely  successful. 


CONCLUSION 

Despite  the  challenges  of  deploying  numerous  complex  systems  and  systems  of  systems,  both  the  1999 
and  2000  AFA  demonstrations  clearly  pointed  out  the  utility  of  networked  distributed  ground-based 
training.  If  the  concept  of  operations  is  to  conduct  coalition  operations,  then  the  challenges  involved 
multi-national  distributed  training  need  to  be  clearly  identified  and  solutions  to  known  problems  tackled 
head  on.  The  AFA  demonstrations  have  shown  that  the  technology  will  support  a  reasonable  number  of 
entities  and  a  level  of  complexity  sufficient  to  augment  peacetime  unit  level  training.  Phase  I  of  Allied 
CRAFT  also  opened  the  technology  door  to  the  Internet  infrastructure  and  proved  that  it  may  be  useful  in 
ways  not  previously  assumed  to  be  possible. 

hi  terms  of  long  haul  experimentation,  the  AFA  2000  proved  that  we  are  not  far  from  being  ready  to 
conduct  coalition  training  on  a  new  level.  Given  the  level  of  complexity  of  the  DMT  demonstrations, 
bandwidth  provided  by  T-l  equivalent  data  lines  is  sufficient  to  begin  now.  No  doubt,  as  the  size  of  the 
synthetic  battlespace  grows  and  the  fidelity  of  the  players  increases,  more  bandwidth  will  be  needed.  But 
the  infrastructure  available  now  and  necessary  agreements  that  have  to  be  initiated  now  can  be  pursued 
now  in  order  to  be  ready  when  the  time  comes. 

It  is  the  opinion  of  the  authors  that  Phase  I  of  Project  Allied  CRAFT  proved  that  existing  technology  is  a 
good  place  to  start,  that  long  haul  network  environments  appear  to  be  feasible,  that  there  is  a  training 
utility  when  using  real-time  virtual  networks  for  coalition  training,  and  that  the  bandwidth  required  and 
latencies  involved  with  transoceanic  lines  are  not  as  bad  as  might  have  been  predicted.  To  be  sure,  there 
is  more  work  to  be  done  in  studying  the  impact  latencies  have  on  training  events.  However,  the 
International  DMT  long  haul  connections  used  in  the  AFA  2000  Technology  Exposition  demonstrated 
that  it  is  feasible  to  expand  the  DMT  concepts  to  NATO  and  our  coalition  partners.  The  significant 
achievement  of  this  collaborative  experiment  is  that  it  will  serve  as  a  first  step  in  establishing  a 
NATO/coalition  DMT  research  environment.  AFRL/HEA  will  continue  to  work  closely  with  the  Defence 
Science  and  Technology  Laboratory  (DSTL),  NATO  countries  and  agencies,  and  our  industry  partners  to 
ensure  that  the  path  to  International  DMT  is  adequately  defined  and  the  technologies  needed  are  identified 
and  developed. 
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8  List  of  Acronyms 


AFA 

AFRL/HEA 

ATO 

AWACS 

C2ISR 

CRAFT 

DIS 

DSTL 

DMT 

FEBA 

ISDN 

JSTARS 

Kbps 

MTT 

NATO 

NES 

PDU 

R&D 

RAF 

SEAD 

SMV 

TTS 

UAV 

UTD 

VSTARS 

WAN 


Air  Force  Association 

Air  Force  Research  Laboratory,  Human  Effectiveness  Directorate,  Warfighter 
Training  Research  Division 

Air  Tasking  Order 

Airborne  Warning  and  Control  System 

Command,  Control,  Intelligence,  Surveillance  And  Reconnaissance 
Coalition  Research  for  Asymmetric  Force  Training 
Distributed  Interactive  Simulation 

Defence  Science  and  Technology  Laboratory  (formerly  DERA) 

Distributed  Mission  Training 

Forward  Edge  of  the  Battle  Area 

Integrated  Services  Digital  Network 

Joint  Surveillance,  Targeting,  Attack  Radar  System 

Kilobits  per  second 

Multitask  Trainer 

North  Atlantic  Treaty  Organization 

Network  Encryption  System  (an  actual  Motorola  product  in  this  paper) 

Protocol  Data  Unit 
Research  and  Development 
Royal  Air  Force 

Suppression  of  Enemy  Air  Defenses 
Space  Maneuvering  Vehicle 

Thales  Training  &  Simulation  (formally  Thomson  Training  &  Simulation) 

Uninhabited  Aerial  Vehicle 
Unit  Training  Device 

Virtual  Surveillance,  Targeting,  Attack  Radar  System  (the  JSTARS  simulation) 

Wide  Area  Network 
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